1. Field
One or more embodiments relate to a system and method of producing a high definition video from a low definition video, and more particularly, to a system and method of producing a high definition video using trajectory estimation and interpolation.
2. Description of the Related Art
Image magnification of a low definition video to a high definition video may be an important function of image equipment, and may be an equally important element that is extensively applicable in everyday life products such as camcorders, or in professional fields such as astronomy, health care, and the like. Recently, there have been an increased number of instances where image magnification have been applied, such as when converting a low definition video, obtained from a digital camera or a camcorder, into a high definition video, reproducing low definition video contents in a high definition display apparatus, or compressing a high definition video into a low definition video and then restoring the compressed low definition video to the high definition video. Accordingly, there may be an increase in a desire for image magnification to obtain high definition video from low definition video.
As one example, image magnification may be implemented using a combination of lenses. However, when magnification is carried out for high magnification using such an optical magnification scheme, this entails a significant increase in volume, weight, cost, and the like. To overcome these problems, image magnification schemes using digital image processing may be used. When using such digital image processing, the image magnification may be desirably implemented using a semiconductor chip, and thereby represent a relatively low cost in comparison with the optical magnification scheme. In addition, when the image magnification is used in conjunction with an existing optical magnification scheme, an image that is more effectively magnified at a high magnification may be obtained. Also, in terms of signal processing, the image magnification scheme may be used to maintain compatibility between systems using images that are different from one another by orders of magnitude, as well as being used to increase a resolution or magnitude of an image. The image magnification scheme may be applicable in reducing the amount of data required for expressing an image in association with image compression.
In terms of the signal processing or image processing, the image magnification scheme may have several approaches. In one approach, an image magnification scheme using interpolation of signals may be easily implemented and practically used. The interpolation of images may be a scheme that may expand intervals between discrete pixels of an original image, and estimate values between the expanded intervals to insert the estimated value into the expanded intervals, thereby obtaining effects in magnifying the image. In this instance, an image magnification method may be determined depending on a manner in which the values between the expanded intervals are estimated, and the magnified image may vary depending on the determined image magnification method.
For example, FIG. 14 illustrates a conventional image processing system where plural frames of a video sequence are input to the multi-frame memory, and more specifically a separate frame memory is required for each frame, e.g., when interpolating pixels within a current frame based on movement estimation from 10 previous frames 10 separate frame memories are required in the multi-frame memory. The multi-frame movement estimation unit 12 estimates movement for each pixel or block of a current frame in relation to the corresponding pixel or block from all previous frames. For example, movement estimation of each pixel may be based on at least 10 previous frames, requiring 10 separate movement estimations based on each of the previous frames for each of the pixels or blocks. From this review of all the previous frames for each pixel or block, an estimation of movement for each pixel or block can be estimated, such that the high resolution interpolation unit 14 can perform an interpolation of pixels or blocks for a current frame.
However, such an approach also requires substantial memory and processing resources, and accordingly results in increased costs, delay, and a limitation of the image conversion to being done in a more substantial memory and processing environment. As noted, a separate frame memory is required for each previous frame, and every pixel or block of the previous frames are stored in each separate frame memory. Further, as movement estimation is required for every pixel or block of a current frame for every previous frame relied upon for movement estimation, substantial processing is required which also requires substantial time. Accordingly, due to such drawbacks, such an image conversion approach cannot typically be implemented in more compact and mobile devices, and typically cannot be performed in a real-time manner. Therefore, there are substantial drawbacks with current image processing approaches for image magnification.